MEMS fast focus camera module

ABSTRACT

An auto focus camera module includes a camera module housing defining an aperture and an internal cavity to accommodate camera module components, an image sensor coupled to or within the housing, a lens barrel within the housing that contains an optical train including at least one movable lens disposed relative to the aperture and image sensor to focus images of scenes onto the image sensor along an optical path, and a fast focus MEMS actuator coupled to one or more lenses of the optical train including the at least one movable lens and configured to rapidly move said at least one movable lens relative to the image sensor to provide autofocus for the camera module in each frame of a preview or video sequence or both.

PRIORITY

This application claims the benefit of priority under 35 USC §119 toU.S. provisional patent application No. 61/657,012, filed Jun. 7, 2012;which is incorporated by reference. This application is also filed asPCT/US13/44844, on Jun. 7, 2013.

FIELD OF THE INVENTION

The invention relates to auto focus camera modules, and particularlyutilizing a MEMS actuator to move one or more movable lenses of anoptical train to rapidly change camera focus distance.

BRIEF DESCRIPTIONS OF THE DRAWINGS

FIG. 1 schematically illustrates a cross sectional view of an example ofan auto focus camera module including a subset of movable lenses and aMEMS actuator in accordance with certain embodiments.

FIG. 2 schematically illustrates another example of an auto focus cameramodule including a different subset of one or more movable lenses and aMEMS actuator in accordance with certain embodiments.

FIG. 3A schematically illustrates a cross sectional view of an exampleof an auto focus camera module including a wire bond image sensorconfiguration in accordance with certain embodiments.

FIG. 3B schematically illustrates a cross section view of an example ofan auto focus camera module including a flip-chip image sensorconfiguration in accordance with certain embodiments.

FIGS. 4A-4B schematically illustrate cross-sectional and top views of animage sensor that is disposed in a recess within a substrate inaccordance with certain embodiments.

FIG. 5 illustrates an example of an auto focus system architecture inaccordance with certain embodiments.

FIG. 6 illustrates a timing diagram for an example of an auto focuscamera module that is programmed to move the subset of movable lensesonce per frame of a preview image stream based on a focus area beingapplied to a central region of a scene in accordance with certainembodiments.

FIG. 7 illustrates certain components of an imaging hardwarearchitecture in accordance with certain embodiments.

FIG. 8 is a plot of contrast value versus focus position for a fulltraversal auto focus mechanism of a camera module in accordance withcertain embodiments.

FIG. 9 is a plot of contrast value versus focus position for a hillclimb auto focus mechanism of a camera module in accordance with certainembodiments.

FIG. 10 is a flow diagram illustrating certain operations of a hillclimb auto focus technique of a camera module in accordance with certainembodiments.

FIGS. 11-12 illustrate effects on lens movements of hysteresis in autofocus camera modules in accordance with certain embodiments.

FIG. 13 illustrates a set of-detected or candidate face locations withina scene and/or on an image sensor or display over multiple preview orvideo frames in accordance with certain embodiments.

FIG. 14 illustrates a set of confirmed or candidate face regions showinga smiley face at the determined location based on a tracking history inaccordance with certain embodiments.

FIG. 15 illustrates overlap of tracked regions—from FIG. 14 with thedetected face locations illustrated at FIG. 13

FIG. 16 illustrates a face tracking technique for an auto focus cameramodule in accordance with certain embodiments.

DETAILED DESCRIPTIONS OF THE EMBODIMENTS

A camera in accordance with embodiments described herein includes animage sensor, which converts an image in an optical domain to anelectronic format, and an optical train that focuses the scene ofinterest onto the image sensor.

Embodiments include cameras configured with an enhanced ability toaccurately capture detail in a scene. The quality of the optical trainand/or the resolution of the image sensor may be selected in accordancewith a desired ability to accurately capture such detail. The imagesensor may contain millions of pixels (picture elements) and the opticaltrain may include two, three, four, five or more lenses.

The position of at least one movable lens of the optical train is notfixed relative to the position of the image sensor, and thus, cameras inaccordance with embodiments described herein can alter the distance fromthe electronic camera at which objects will be in focus on the imagesensor. A system may be utilized in accordance with embodiments todetermine one or more distances of one or more principal objects in ascene from the camera. The at least one movable lens is movable inaccordance with the determined distance and/or until one or moreprinciple objects are in focus on the image sensor. These objects canrange from being very close (10 cm or closer) to very distant (infinity)from the camera.

Embodiments are provided herein of cameras that provide image qualitythat is better than conventional autofocus and fixed focus cameras.Cameras in accordance with certain embodiments also exhibit miniaturesize, as well as advantageous power efficiency.

Electronic cameras in accordance with certain embodiments exhibit anadvantageous capability to alter the field of view significantly. Forexample, a photograph of a family taken in front of their house mightinadvertently include a refuse container at the edge of the scene when aconventional camera is being used. A camera in accordance with certainembodiments can be adjusted to restrict the field of view of the camerato eliminate this artefact from the captured image. Conversely, aphotograph of a family taken on top of a hill can be enhanced using acamera in accordance with certain embodiments by adjusting to a widerfield of view that captures more of the panorama.

Cameras in accordance with certain embodiments exhibit clearimprovements in overall performance by incorporating dynamic field ofview feature with an auto focus mechanism. In certain embodiments, thedesign of the optical train of the camera includes a part that is fixedand a part that is movable along the optical axis of the camera by anactuator. In certain embodiments, some image processing is provided bycode embedded within a fixed or removable storage device on the cameraand/or using a remote processor, e.g., removal of image distortion.

Advantageous cameras are provided in accordance with certain embodimentsthat integrate all three of these in a compact camera module. Suchcamera module may be a stand alone camera product, or may be included ina fixed or portable electronics product, and/or in various otherenvironments such as automobiles.

Several embodiments will now be described with reference to the figures.Electronic cameras are provided herein that advantageously incorporateintegrated auto focus and optionally zoom functionality. In certainembodiments, the autofocus and zoom functions utilize a combination ofan advantageous optical train and processor-based image processing, andin certain embodiments include the same or similar components in bothcases.

Alternative approaches to add auto focus may involve moving one or moreother lenses in the optical train as a group. An auto focus zoom camerabased on this principal of operation is described in U.S. Patentapplication Ser. No. 61/609,293 which is incorporated by reference. Thismovable lens group may contain more than one movable lens, and maycontain four lenses as described in the '293 application, as well asvarious numbers of stops and apertures depending on the particularnumber and geometry of the lens or lenses forming the movable lensgroup.

An optical train in accordance with certain embodiments that includesauto focus, and optionally also zoom, includes two general components.FIG. 1 illustrates an auto focus zoom camera module including a firstmovable group L1-L4 in the example of FIG. 1, including one or morelenses that can be moved along the optical axis of the camera, and afixed lens group, L5 in the example of FIG. 1, that includes at leastone lens that is fixed in position. The moving lens is the lenses in theexample of FIG. 1 are closest to the scene and a fixed lens is closestto the image sensor.

In general terms, the moving lens group performs the function ofaltering the focal distance of the camera, and in embodiments of cameramodules that also include zoom, at least one fixed lens is configured toperform the optional electronic zoom function of matching the PSFfunction of the optic to the imager and compensating for the fieldcurvature induced by the moving lens group. The fixed lens that mayperform this function in specific embodiments described in the '293application is the lens closest to the image sensor. At least one movinglens is located at an appropriate distance along the optical axis toachieve the desired focus distance, while at least one fixed lens islocated such that its back focal length matches the distance between thelens and the imager.

A processor programmed by embedded code collects information from pixelsin the image sensor and makes changes to the associated electronic file,in some cases automatically and in others based on user inputs. Forexample, the degree of zoom is adjustable. The processor endeavors tocorrect for distortion and other artefacts that are produced in apredictable manner by the optical train. The image processing featurescan be implemented in either hardware or software. In certainembodiments, these features are placed early in the image processingpipeline, such as RTL (resistor transistor logic) code embedded in theimage sensor, while in others they are placed on an external DSP(digital signal processor) or entirely in software in a processor, suchas the base band chip in a mobile phone.

An auto focus zoom camera example in accordance with the exampleillustrated at FIG. 1 has a focus distance that can range 10 cm to 9 m,is typically 15 cm to 5 m and is preferably 20 cm to 3 m (excluding thehyper-focal distance), while the zoom function can range x0.5-x5, istypically x1-x4 and is preferably x1-x3. A noteworthy characteristic ofthe final electronic file produced by an advantageous camera inaccordance with certain embodiments is that file size and effectiveresolution of the image contained within it may be largely constant incertain embodiments irrespective of the focus distance and zoom setting.

In another embodiment, an auto focus camera may have an entire opticaltrain that is moved in an auto focus process. In addition, advantageouscameras in accordance with embodiments described herein that includeoptical trains with both a movable component and a fixed component maybe configured in accordance with many other examples than the twoillustrated at FIG. 1 and FIG. 2. These advantageous auto focus zoomcameras have one or more parts of the optical train fixed and one ormore parts moving. In certain embodiments, cameras exhibit exactitude ofcentering and tilt alignment of the moving lens to the fixed lens thatdiffers from conventional fixed or auto focus cameras.

FIG. 2 illustrates another example of an auto focus camera modulewherein the middle lens L3 is movable between two pairs of fixed lensesL1-L2 and L4-L5. This embodiment is described at U.S. patent applicationSer. No. 61/643,331, which is incorporated by reference. The embodimentswherein only a single lens is included in the movable lens group, suchas the middle lens L3 being movable relative to two pairs of fixedlenses L1-L2 and L4-L5 located on either side of the middle lens L3 havean advantage of small mass and thus a relatively low force is involvedin moving it, and even has a surprising further advantage that a smalldisplacement range actuator may be used. By moving the middle lens inthe optical train in certain embodiments, e.g., L3 in an optical trainincluding five lenses or L4 in an optical train of seven lenses or L2 ina train of three lenses. In other embodiments, the movable lens isoffset from the middle somewhere between at least one fixed lens and therest of the optical train, e.g., L2 or L4 in the five lens embodiment orL2, L3, L5 or L6 in the seven lens embodiment. Still other embodimentsinvolve movable lenses at one or both ends of the optical train.

Contrary to perceived expectation, it transpires that to achieve asimilar focus range to a conventional auto focus camera, the middle lensin the example of FIG. 2 is moved a relatively short distance, typicallyaround 100 um. This makes possible the use of novel forms of actuator,such as MEMS, to move the lens and a number of consequential benefitsarising from the inherent characteristics of such devices. Of the manybenefits of this design, small size, low power consumption, low noise,high speed and high accuracy of movement and other improvements areprovided.

MEMS Actuator

A MEMS actuator is coupled to L3 in the example of FIG. 2 (and to themovable lens group L1-l4 in the example of FIG. 1) to provide auto focuscapability in certain embodiments. In other embodiments, a voice coilmotor (VCM) or a piezo actuator may be used to provide movementcapability.

Suitable MEMS actuators are described in several of the US Patents andUS patent applications incorporated by reference herein below, e.g., seeU.S. patent application Ser. No. 61/622,480. Another MEMS actuatorhaving a somewhat different design is described in US-PCT applicationno. PCT/US12/24018. Both of these US patent applications areincorporated by reference, and other examples of MEMS actuators andcomponents thereof are cited and incorporated by reference below asproviding alternative embodiments. Such actuators can be fabricated insilicon or substantially polymeric materials and have a stroke of around100 um. They also exhibit a number of other beneficial characteristics,which are conferred on an auto focus zoom camera module of the typedescribed. These include, very low power consumption, fast and preciseactuation, low noise, negligible particulate contamination and low cost.

A MEMS actuator in accordance with certain embodiments may be thought ofas generally a unidirectional device, setting aside for the moment anycentering or tilt alignment movements that may be ascribed to anactuator component. That is, a MEMS actuator in accordance with certainembodiments has a rest position and the actuator can be driven from thatrest position in one dimension only. This has a benefit for the assemblyof auto focus camera modules in that it permits the entire lens train,or a substantial portion thereof, to be assembled as a pre-alignedunitary component. For subsequent assembly and calibration steps, it canthen be handled similarly to or in exactly the same manner as the lenstrain of a fixed focus camera, namely the focus can be set by insertinga holder, containing the lens train into a sleeve fixed over the imagesensor. In certain embodiments, the holder and sleeve are coupled by ascrew thread.

FIG. 2 also schematically illustrates a cross-section through an autofocus zoom camera in accordance with certain embodiments that utilizesassembly with the lens train fabricated as a pre-aligned unitarycomponent. The image sensor 201 resides on a substrate 202 to which isattached a sleeve 203. The sleeve has a screw thread 204 in the exampleillustrated at FIG. 2. The holder 205 containing the lens train 206 hasa mating screw thread 207. Rotating the holder with respect to thesleeve moves the entire lens train, in this example embodiment, alongthe optical axis 208 of the camera, permitting the focus to be set.Alternatives to the matching screw threads 204 and 207 include matchinggrooves and lands in various patterns permitting focus to be setcontinuously or discretely such as with a series of notches,spring-loaded pins or levers or elastic materials or other techniques tocouple the lens train holder 205 with the sleeve 204 in a way thatallows the distance between the image sensor 201 and one or more fixedlenses of the lens train 206 to be set.

A precision alignment in accordance with certain embodiments of theoptical train permits transmission of images at high fidelity. Certainembodiments involve alignment of the various elements of the train,principally the lenses, with respect to tilt, centering and rotationwith respect to one another to a certain degree of accuracy. While it ispossible to achieve very exact alignment of one lens to another usingactive alignment techniques in certain embodiments, passive methods areused in certain embodiments, and typically wherever possible, due to thehigh speed of assembly and low cost of this approach. In the auto focuszoom module of certain embodiments, passive alignment tolerances areaccommodated in all but one of the joints of the lens train.

Camera Module with Protective Cover

In certain embodiments, an optical surface can be added to the imagesensor as a singulated component. This optical surface can serve as acover, made of transparent glass or polymer, to prevent dust or othercontaminant from the reaching the active surface of the sensor, whilepermitting visible light to get through to the sensor. The opticalsurface can also serve as an infrared (IR) filter, particularly for asilicon sensor. An IR absorbing material may be used for the cover or anIR coating may be applied to the glass or polymeric or other opticallytransparent protective cover. The optical surface can also be formed toprovide optical power such as in the shape of a replicated lens. Aprocess for forming the singulated component at the wafer stage beforedicing is provided hereinbelow.

The component includes an active image sensor protected againstcontamination using wafer level hybrid optics. This approach has anotheradvantage in that an overall physical Z height of the camera module maybe reduced by incorporating such hybrid optics with the camera modulecomponent.

The active image area on the image sensor is protected in accordancewith certain embodiments at the wafer stage before dicing or singulationof the image sensor wafer into discrete dies. This protection of theactive image area is achieved in certain embodiments by attaching aglass wafer, such as a blue glass or IR coated glass, or other materialsuch as a polymer or other material that is transparent to visible lightand absorbs or otherwise blocks IR light. Further improved functionalityof this glass protection is achieved by adding a wafer level opticselement.

FIG. 3A schematically illustrates an example camera module that includesa wire bond coupled to the camera module component. FIG. 3Bschematically illustrates an example camera module that includes aflip-chip. The example camera module illustrated schematically at FIG.3B may use thermal compression or a thermosonic process. These aredescribed in more detail at U.S. patent application Ser. No. 13/445,857,which is incorporated by reference.

In auto focus and optional zoom camera modules in accordance withvarious embodiments, processor-based components such as distortioncorrection components, chromatic aberration correction components,luminance, chrominance, and/or luminance or chrominance contrastenhancement components, blur correction components, and/or extendeddepth of field (EDOF) and/or extended or high dynamic range (EDR or HDR)components.

Another example is illustrated schematically at FIG. 4A and FIG. 4B, andis also described in detail at the Ser. No. 13/445,857 US applicationincorporated by reference above. FIGS. 4A-4B include structuralcomponents illustrated in section and plan view, respectively. A flatsubstrate forms the base of the camera module of FIGS. 4A-4B. A purposeof this substrate is to provide structural support, and so suitablematerials include metals (e.g., titanium), ceramics (e.g., alumina) andhard polymers like Bakelite. The substrate material may be moulded orone or more other methods may be used to fabricate an array ofthrough-holes in it. In certain embodiments, these through holes willeventually be fully or partially filled with conductive material as partof the structure that provides the electrical interface to the cameramodule. Because the substrate contributes to the overall height of thecamera module, it is a very thin yet sufficiently rigid. The mechanicalproperties of the material of the substrate, including its modulus andfracture toughness, are carefully selected in certain embodiments. Thesubstrate may be around 200 microns thick, and can have a thickness bein a range between approximately 50 microns and 400 microns.

The image sensor and cover glass are coupled over roughly a centralportion of the substrate in the example embodiment illustrated at FIGS.4A-4B. The image sensor may be attached to the substrate using adhesivebonding or magnetically, or using one or more clips or complementaryslide or twist fastening components, or using fit bonding utilizingstatic adhesion or thermal or compression shrink or expansion fitting,or otherwise. Over a substantial portion of the remainder of thesubstrate, in this example, is attached a flexible circuit. The methodof attachment may be adhesive bonding or one of the just mentionedmethods or otherwise. The flexible circuit may include in certainembodiments thin conductive tracks made of copper or other metal orconducting polymer on the surface of and/or embedded within a softpolymeric material like polyimide. Apertures or other features may beused to provide access to the copper tracks to make electricalconnections.

As illustrated in the example of FIGS. 4A-4B, the flexible circuit hasan aperture that is smaller than the image sensor in plan area. Thispermits the flexible circuit to be placed over the image sensor, suchthat the bond pads on the image sensor are covered by the flexiblecircuit. In this way, electrical joins may be made between the bond padson the image sensor and suitable lands on the flexible circuit. A widechoice of methods and materials are used in accordance with severalembodiments to effect such joins, with examples including conductiveadhesives, thermo-compression bonds, soldered joints, and ultrasonicwelds.

The image sensor is connected or connectable electrically to theflexible circuit, enabling tracking on a flexible circuit in accordancewith certain embodiments to be used to route electrical connections toother sites, which may include active and/or passive components. Activeand/or passive components can be attached and interconnected to theflexible circuit in various embodiments using established methods andtechniques. In FIGS. 4A-4B, three (3) passive components are included inthe camera module, along with ten (10) bond pads and eight (8)through-hole solder interconnects, but these numbers and locations andshapes and sizes are provided by way of illustration and many variationsare possible.

External electrical connection to the camera module involves in certainembodiments electrical connection to suitable lands on the flexiblecircuit. By design, these lands are advantageously located over thethrough holes in the substrate. Although FIGS. 4A-4B depict pillars ofcopper for these electrical interconnects, the electrical interconnectscould be fabricated from a variety of materials and structures includingsolder pillars, stacked stud bumps, conductive adhesives and/or deepaccess wire bonds. Other embodiments include mechanical structures likesprung elements and pogo pins. Where solder pillars are used, on reflowof the solder, the periphery will change shape into a hemisphere so thatthe external interface of the camera module resembles an interconnectfor semiconductor packages similar to a ball grid array. The examplestructure shown in FIGS. 4A-4B includes a flexible circuit that has aslight bend, while in other embodiments, the flexible circuit does nothave a bend.

FIGS. 4A-4B schematically illustrate an image sensor that is disposed ina recess in the substrate, such that image sensor bond pads are on thesame level as the underside of the flexible circuit, although in otherembodiments, there may be an offset. Some adjustment to the detail ofthis alignment may take into account the thickness of the joining mediumused to attach and connect the flexible circuit to the bond pads.

MEMS Fast Focus

Auto focus (AF) can be sub-divided into at least two types: one beingactive AF which typically involves additional parts like as IR LED andsensor, and another being Passive AF or TTL AF (Through The Lens AF),which uses the actual image acquired by the sensor to detect objectdistance. Auto focus camera modules in accordance with certainembodiments include compact digital still cameras and mobile phonecameras that use a “Contrast Measurement Method” which is a sub-type of“Passive AF”, while other methods are used in other embodiments. FIG. 5illustrates schematically combined hardware or H/W (lower) and softwareor S/W (upper) in a system architecture of an auto focus mechanism.Software is typically implemented in ISP.

A mobile phone camera may use a CMOS sensor as an image sensor. A CMOSsensor may use rolling shutter during preview. After finishing framedata transfer and calculating statistics for AF of the frame, the systemmay provide data to an AF component or algorithm or application, thatincludes a callback feature. In a callback interface, the AF componentevaluates image contrast, decides next focus position (video) or finalfocus position (still image capture), and controls the actuator via lensdriver software task (that programs the lens driver IC).

Advantageous auto focus camera modules with MEMS actuation in accordancewith certain embodiments are fast enough to move the lens within asingle frame acquisition cycle without interfering with the exposure anddata-transfer from the sensor. At HD data rates, 80%+ of the image framecycle (33 ms) is needed to move data from sensor to ISP and onto themain system memory.

MEMS actuators that may be used in various embodiments are described atU.S. Ser. No. 61/622,480 and PCT/US12/24018, which are incorporated byreference, and other embodiments of MEMS actuators and camera moduleincluding MEMS actuators are described in references cited andincorporated by reference hereinbelow.

An advantage of MEMs is its quickness as illustrated in the timingdiagram of FIG. 6. FIG. 6 illustrates a timing diagram for an example ofan auto focus camera module that is programmed to move the subset of oneor more movable lenses once per frame of a preview image stream based ona focus area being applied to a central region of a scene in accordancewith certain embodiments.

The MEMS actuator is fast enough, in combination with an exposure timebeing shortened during AF as well as a data transfer frame rate alsobeing fast, so that AF achieves both good speed and accuracy within asingle frame cycle (<33 ms for 30 fps) in auto focus camera modules inaccordance with certain embodiments.

The “focus area” is shown in FIG. 6 as being applied to the centralregion of the image (orange/brown region). Thus in this example the lenscan still be moving during the first 20% of the acquisition cycle forthe next image frame (top yellow regions). The lens can also beginmoving during the last 20% of the acquisition cycle (bottom yellowregions) because these initial and final lines of the image data are notused to determine focus in certain embodiments.

The image may be initially partially blurred or distorted due to lensmotion. However, the AF component of camera modules in accordance withcertain embodiments advantageously determines its “sweet spot” within ahandful of image frames and is stable thereafter.

The MEMS is so fast, that in certain embodiment under certainconditions, the AF camera module is not refocused on every image frame.In other embodiments, where the lens is to be moved on every imageframe, some additional smart processing is included that avoids anyunpleasant “focus hunting” where the focus could otherwise be constantlyoscillating due to being overly sensitive.

Relationship Between Sensor+AF, ISP, System Memory and CPU/GPU

FIG. 7 illustrates certain components of an imaging hardwarearchitecture in accordance with certain embodiments. AF statistics andalgorithm component may be typically implemented in the ISP and in thoseembodiments the ISP provides instructions to the AF hardware to drivethe lens to its updated position. The ISP is designed to process sensordata as it passes through to the main system memory The data canalternatively be passed to a Host memory, e.g., via a MIPI I/F or theISP can look to the Host as a sensor, e.g., for a sensor parallelinterface, in certain embodiments.

The ISP is linked in certain embodiments to the image sensor by adedicated high-speed bus and the ISP buffers sensor data in a multi-linebuffer. Image data may be processed by the ISP as it passes through thisbuffer architecture and statistical data is accumulated and/or retainedwithin ISP for the current image frame. ISP may have some retained dataon past image frames, but this is typically high-level statistical dataor image characteristics determined from these.

ISP also modifies raw image data, e.g., to de-Bayer the R-G-G-B sensordata, converting to RGB or YCC format. ISP may also adjust imagesharpness, white balance, color balance and perform some limitedfiltering such as noise suppression, gain adjustment and adaptivefilters. The ISP may also implement other high level image processingand analysis functions such as face detection and tracking, and smileand blink detection and/or other techniques described in patents andpatent applications incorporated by reference below.

AF Algorithm Layer

Camera modules in accordance with certain embodiments utilize “SingleAF” or “Continuous AF.” Single AF techniques involve one AF operationwith a certain trigger such as a shutter for still aquisition.Continuous AF techniques involve performing AF continuously duringpreview or video capture.

Full Traversal and Hill Climb AF

Full Traversal AF techniques scan an entire focus range (e.g., using afull traversal of a movable lens or lens barrel of a camera module). Anexample of a timing diagram for a Full Traversal AF technique isprovided at FIG. 8. A contrast value of a region of interest or ROIwindow from infinity to macro is determined. The lower, longer blackline with arrow pointing to the right represents the full focus range oftravel of the focusing lens, lenses or lens barrel of the auto focuscamera module in this embodiment. A peak focus measure is found. Thelens, lenses or lens barrel is moved back to focus position with optimalfocus measure (typically the peak contrast value). This movement backfrom macro to the position of optimal focus or peak contrast isillustrated by the upper, shorter black line with arrow pointing to theleft in FIG. 8. The total distance traveled by the lens during the Fulltraversal AF process can be estimated as the sum of the distancesrepresented by the two black lines multiplied by the average speed. Thefull traversal AF technique can take more time to move the lens, lensesor lens barrel than a technique that utilizes less than the entire focusrange, although finding the peak is often more straightforward than withother approaches. Because there is a risk that objects and people in thescene may change during the focusing cycle, it is desired in certainembodiments to reduce focusing time.

A Hill Climb AF technique may be adopted in certain embodiments toreduce focusing time compared with the Full Traversal AF. An example ofa timing diagram is provided at FIG. 9 to illustrate a Hill Climb AFtechnique. In the Hill Climb AF technique, image contrast is checked fora region of interest window from Infinity, moving back toward macroposition. Once the algorithm detects a contrast decrease, which itshould do just after the lens, lenses or lens barrel crosses theposition that produces peak contrast and after the lens, lenses or lensbarrel has traversed only a fraction of the entire focus range asrepresented by the lower black line with arrow pointing to the right inFIG. 9, then it steps back towards the peak as represented by the upper,short arrow pointing to the left in FIG. 9. The first steps through thefocus range are relatively large in certain embodiments. After contrastdecrease is detected a smaller step size is used. Even with a somewhatmore complicated set of operations, focusing time is typically reducedwhen compared with full traversal AF, because a shorter distance istraveled by the lens during the Hill Climb AF technique than for theFull Traversal AF technique, as evidenced for example by the sum of thetwo black lines being far smaller in FIG. 9 than in FIG. 8. As long asthe time is not increased too significantly by slowing the movementaround the peak contrast position or by adding too much time to counterhysteresis (as described below), the hill climb AF technique can be usedin advantageous combination with a MEMS-based auto focus camera module,optionally also having zoom. The embodiment illustrated at FIG. 2 may beselected when very fast auto focus is desired, because the singlemovable lens L3 in that embodiment has only a relatively short distanceto traverse, as mentioned above. Many variations on the Hill Climb AFtechnique may be used in various embodiments with improved focusmeasures, or using data from a small number of initial AF steps to findthe peak value using smart interpolation techniques such as bi-cubic orspline interpolation.

FIG. 10 includes a flowchart structure of an example of a Hill Climb AFtechnique that may be used in certain embodiments. A focus lens orlenses are moved to the extreme of a focus range of the camera. Contrastdata are obtained from the image signal processor or ISP and saved to anarray. For this example, it is assumed that region of interest or ROI(AF window) has been fixed. The latest frame data are compared withprevious data. A determination is made as to whether a peak has beenpassed over. Contrast data have noise, so it may not show the peak thatjust the latest contrast data are smaller than ones of a previous frame.So, detecting the peak may involve using a threshold or checking ahistory of contrast data. If it is determined that a peak has beenpassed over, then the focus optic(s) is moved to the peak position andthe AF ends. If it is determined that a peak has not been passed over,then it is determined whether current position is macro. In other words,the processor programmed by the AF algorithm could not find a peak. Thefocus length can either be moved to a next position, or if no peak isfound over an entire focus range, then the system can retry the AFtechnique from the beginning, or the focusing optic(s) can be moved to adefault position, e.g., hyper focal length.

Hysteresis

Hysteresis is illustrated in FIG. 11 and is defined as a difference inlens position when a same voltage is applied on an actuator when movingthe movable lens, lenses or lens barrel from macro to infinity positionand then in the opposite direction from infinity to macro position.Notably, MEMS actuators that are included in auto focus camera modulesin accordance with certain embodiments include three microns or lesshysteresis. In comparison, VCM actuators often have twenty microns ormore hysteresis, which is so much hysteresis that an AF technique mayadd a step of going back over the peak, and moving to peak again afterfinding the peak as in the illustrative example of FIG. 12. This adds alittle distance to the overall sum by adding a third movement backtoward macro as illustrated by the upper, shortest of the three blackarrows in FIG. 12 to the sum that was determined of only the other twoarrows with reference to the example timing diagram of FIG. 9, butincreases the difference compared with VCM auto focus due to the greaterdifficulty involved in finding the contrast peak with sufficientprecision in that far higher hysteresis environment.

The basic concept of Continuous AF is approximately the same as SingleAF. However, unnecessarily frequent re-focusing should be avoided asproblematic, particularly for video acquisition. A threshold can bepredetermined wherein small enough variations in the scene or the focuswindow would not trigger re-focusing, while if the algorithm issensitive, it can be changed on each image frame. Accordingly, somealgorithm adjustments are provided in certain embodiments of ContinuousAF techniques to achieve certain desired results.

Scene Change Detection

A scene change detection component may be used in a Continuous AFtechnique in certain embodiments that can serve to avoid frequent focusmovements. This can set a threshold for the change in scene before newfocus movements are requested.

Predictive Face AF

In certain embodiments, information derived from detected face images ina scene are used to assist in providing estimates of face distances andface locations in an image frame. Face-based techniques are combinedwith MEMS AF techniques, particularly as hysteresis is relatively lowand repeatability is high with auto focus camera modules that includeMEMS actuators. If integration with face-tracking is implemented thensmarter algorithms are possible.

Additional information on predicted face regions within image frames iscoupled with statistical face tracking data in certain embodiments andcan be used to inform the AF algorithm, when tracked face regions areavailable. This also enables accurate scene change detectionparticularly when people and/or faces are often the main source of scenechanges. As an example, eye-to-eye distance changes can be determinedfrom frame-to-frame, providing both distance and velocity metrics for atracked face region. This information helps avoid focus searching in thewrong direction. Where multiple face regions are tracked there arevarious options, e.g., focus may be based on the closest face, the mostcentral face, a longest tracked face, an average across multiple faces,or other preset criteria. With a touch screen interface, in certainembodiments, the user may select a face region to prioritize.

Hardware Face Tracking within ISP

Hardware add-ons for ISP may be implemented which can extract localimage primitives and apply multiple parallel object classifier templatesto buffered image data as it passes through the ISP (see, e.g., USpublished patent applications serial numbers 20120120304, 20120106790,and 20120075504, which are incorporated by reference).

In one embodiment, potential (rectangular) face regions are determined.This data is written with an original image frame to system memory forsubsequent processing by a main system CPU or GPU. This data is retainedin certain embodiments and processed statistically over multiple imageframes at ISP level, or within a hardware “add-on” to the ISP. This isused to provide a hardware face tracking mechanism in certainembodiments which can determine predicted locations of a face in laterimage frames. As the main CPU typically will configure a local hardwaremodule from time to time, e.g., to assist high-level face analysisfunctions by loading different classifier templates in certainembodiments, it is possible to obtain occasional confirmations fromother algorithms running on main CPU/GPU. Classifier templates are notrestricted to faces. Classifier templates have been successfullyimplemented using the same architecture for eye-regions, hands, facialexpressions (smile, blink) and for non-human objects including vehicles,animals, buildings and trees.

One particularly relevant classifier for certain applications is the“eye-pair” template to confirm a face region and provide, in addition, amore accurate indication of the size of the face region.

Referring now to FIG. 13, a set of detected face locations areillustrated as included rectangles within a large rectangular windowthat may represent an image boundary. These included rectangles are theresult of matching multiple short classifier chains in parallel to awindow of an image frame. The rectangular regions show where a potentialface region has been detected when each of the illustrated rectangularwindows successfully matches one of multiple short classifier chainsapplied in parallel to that window of the image frame. Regions withmultiple, overlapping windows have higher probability of containing aface than regions with only one window or with fewer windows. In theexample of FIG. 13, a significant number of single detections are noted,although these are unlikely to indicate a real face region if no othermatches are determined at that location, because the individualclassifier chains are designed in this example embodiment to be highlyinclusive for a specific set of face characteristics, e.g., pose,in-plane rotation, and/or illumination.

Over multiple image frames, confirmations may be obtained for a set ofcandidate face regions and a history may be built. The history mayinclude a recorded direction and degree of motion of a confirmed region.Thus, at the beginning of an image acquisition, the ISP can be informedas to where face regions were in a previous image frame or frames. Thesemay have just been analyzed and an output similar to FIG. 13 may beavailable for the last frame. The ISP can also keep a statisticalhistory of tracked face or other object regions in certain embodiments,as illustrated in the example of FIG. 14.

FIG. 14 illustrates confirmed face regions showing at the locationswhere “smileys” appear within the large rectangular window based incertain embodiments on tracking history and/or a combination of trackinghistory and one or more additional criteria. Such additional criteriamay include non-real-time confirmation from a more sophisticated andperhaps slower algorithm running on a main CPU. A focus distance orfocus value for each of these faces or other objects or regions ofinterest (ROIs) can also be determined and recorded, e.g., as f1, f2,f3, and f4 in FIG. 14. The dashed lines in FIG. 14 indicate a predictedface region or a location where a face is expected to be in the imageframe currently being acquired. This information may be based partly onhistorical tracking data, partly on determined current direction and/orspeed of motion, and/or partly on the size (closeness to camera) of thetracked face. Note for example that f4 is moving with reasonable speedin the horizontal direction, while f3 is moving at a slower speed in theopposite still horizontal direction so the predicted face region is lessextended in the direction of motion for f3 than for f4. FIG. 14 alsoindicates that f1 is moving slowly in the vertical direction, and thatno movement is detected for f2 or that any movement detected for f2 isbelow a threshold minimum.

One aspect of face-based detection is that a MEMS-based auto focuscamera module in accordance with certain embodiments can advantageouslyuse one or more predicted face regions as ROIs for the AF algorithm. Inthe example shown in FIG. 14, there are four (4) regions eachpotentially with at least a slightly different focus setting, i.e., f1,f2, f3 and f4. A range of different approaches may be applied. In oneembodiment,—focus position is based on a single value determined fromall detected faces or a subset of detected or candidate face regions. Inanother embodiment, focus is determined based on or set to focus on aclosest (largest) face; or on a most distant (smallest) face; or on aface located at a focus position which corresponds to an intermediatesetting between closest and most distant. In another embodiment, thefocus may be time based, e.g., the focus may be set to a most recentlydetected face; or on the longest detected face, or on a face that hasmaintained face-lock during tracking for a threshold time also withsufficient stability, or other parameters, criteria or combinations

FIG. 15 illustrates an overlap of the tracked rectangular regions havingsizes in the directions of movement of the respective faces that scalewith the speed that the face is moving within the scene from FIG. 14with the multiple, overlapped in probable cases, single-frame detectionsof FIG. 13. Interestingly in this example, a somewhat likely new faceregion appears to have entered the image field from the right-side thathas four single frame detection corresponding to matching four differentshort classifier chains, even though a face tracking lock is notindicated as having been established, e.g., by providing a dashedrectangle that scales with movement direction and speed, such that theremay be no statistical data, nor associated predicted regioncorresponding to that location.

Specialized hardware is provided in certain embodiments to filterobjects such as faces, and/or that can be trained to detect any of awide variety of objects and object types. The templates encode a numberof object features that may describe only loosely the object, e.g., tobalance a desire not to miss candidate objects and to run the processquickly. A second processing step may be performed by the SW usinghigher quality templates on any candidate regions. For example, thehardware templates can be trained with most commonly interesting objects(e.g. buildings, cars, people, pets, favorite sports teams, etc.) andthe output from the object matching HW block can be used as input intothe AF algorithm (as ROIs). In fact in certain embodiments, these ROIsare utilized as being very suitable for computing also certain chromaticmeasurements that could help computing approximate distances to each ofthose individual ROIs and thus helping with fast positioning of thelens.

ISP-Implemented Face Focus Techniques

In certain embodiments, an image signal processor or ISP isadvantageously programmed to control an AF algorithm, so that theexposure cycle of the first pixel of the first predicted face region isneither blurred nor distorted by lens movement, or at least any suchblurring or distortion is minimized or kept small. ISP control of theexposure cycle of an image sensor may involve responsibility to reset asensor rolling-shutter at a correct time prior to start of a next datatransfer cycle. When implementing face based AF, an ISP in accordancewith this embodiment may also have knowledge of predicted face regionsfor a current image frame and may also control lens motion via an AFalgorithm.

The ISP determines in certain embodiments when the first pixel of thefirst face region will be reset and ensures that lens motion does notoccur during its exposure cycle. If the determined lens motion cannot becompleted within this timeframe, an exception is triggered to the AFalgorithm or task so that it is aware that an optimal lens position wasnot achieved. In one embodiment, an ISP provides a synchronizationmechanism that is implemented such that the sensor I/F in the ISPsignals to the rest of the ISP (either HW blocks or onboard CPU) whenthe ROIs exposure started and when it has been completed. The ISP shouldtake care not to allow LENS movement during the exposure of the AF ROIs.New MEMS optimized ISPs are provided in certain embodiments thatimplement this signaling mechanism.

Lens motion ceases during that portion of image acquisition during whichpredicted AF ROIs are acquired. Once the last pixel of the lastpredicted ROI is acquired, then the ISP may re-initiate lens motion incertain embodiments. Typically, however, the AF algorithm or taskprograms the ISP to first finish both computing focus metrics on each ofthe detected face regions and determining an overall focus metric. Basedon this new focus metric, the ISP decides if lens position is notoptimal and should be further adjusted.

Depending on one or more timings of the exposure cycle and the templatematching cycle in certain embodiments, the AF algorithm may program theISP to wait for additional confirmation that predicted face regions did,in fact, contain a face during the current cycle before deciding if afocus adjustment is to be initiated. In some embodiments, a furtherdelay may be implemented when a newly detected face region waits to beconfirmed by more sophisticated CPU/GPU algorithms, or one or moreadditional image frames are used by the ISP to enable statistical datato be established on a new region.

An example of a flowchart of a face-based autofocus algorithm inaccordance with certain embodiments is provided in FIG. 16.

This algorithm is based on a software embodiment of the face trackingalgorithm, but it can be seen how this can easily be replaced by thehardware template matching module and ISP-level firmware to speed up theface information and eye-distance components.

Re-Focus within a Single Image Frame

The speed of MEMs not only enables re-focus from frame to frame, butalso allows re-focusing within a single frame in certain embodiments.Blur or distortion to pixels due to relatively small movements of thefocus lens are manageable within digital images. Micro-adjustments to AFare included in certain embodiments within the same image frame serving,e.g., to optimize local focus on multiple regions of interest. In thisembodiment, pixels may be clocked row-by-row from the sensor and sensorpixels may correspond 1-to-1 with image frame pixels. Inversion andde-Bayer operations are applied in certain embodiments.

Referring back to FIG. 14, four horizontal lines are indicated thatrepresent lines of pixels and terminated each with an arrow-headindicating these pixels are heading for the ISP after they are clockedfrom the sensor in sequence. Pixels are clocked out row-by-row from thetop down and from left to right across each row in certain embodiments.Looking to the top row, pixels to the left of the first predicted faceregion (f1) are ‘clear’, whereas pixels to the right of the first pixelof this ROI are blue/dark. Lens motion is ceased during the exposureinterval of these ‘dark’ pixels to avoid lens-motion blur/distortion.The lens remains still while all intermediate rows of the sensor down tothe last pixel of the second face region (f2) are exposed in thisexample. However, once the last data pixel of f2 is clocked to the ISP,the lens could begin to move again, although the lens motion would beceased again to allow the first pixel of the third face region (f3) timeto complete exposure. Thus if the time for two exposure intervals islonger than the time gap to offload data from f2 to f3, there will notbe sufficient time for lens motion between f2 and f3. The physicaloverlap of rows of f1 and f2, and also of f3 and f4, in the example ofFIG. 14, does not allow any lens motion between these ROIs. Re-focuswithin a frame may be provided in certain embodiments when the exposuretime of individual pixels is quite short compared with the full imageacquisition cycle (e.g., 33 ms).

Alternating Focus Techniques

In another advantageous embodiment, focus is switched between faceregions for alternating image acquisitions. In an example of thisembodiment, the lens may be moved to an intermediate position that liesapproximately midway to the four focus settings, f1, f2, f3, and f4 inFIG. 14. Then, on each successive image frame the focus is moved to theoptimal focus for each face region. This cycle is continued onsubsequent image acquisitions.

The resulting image stream has a sharp focus on one of the four faceregions in successive image frames while other regions of the image areless sharply focused.

US published patent applications nos. US20110205381, US20080219581,US20090167893, and US20090303343 describe techniques to combine one ormore sharp, underexposed images with one or more blurred, but normallyexposed images to generate an improved composite image. In this case,there is one sharply focused image of each face or other ROI and threemore or less slightly defocused images of the face or other ROI. Incertain embodiments, an improved video is generated from the perspectiveof each face or other ROI, i.e., with each face image in optimal focusthroughout the video. One of the other persons can change theconfiguration to create an alternative video where the focus is on theminstead.

In another embodiment, a similar effect is obtained by using two camerasincluding one that is focused on the subject and one that is focused onthe background. In fact, with a dual camera in accordance with thisembodiment, different focus points are very interesting tools forobtaining professional depth 2D video footage from an ordinary or evencheap 3D camera system (e.g., on a conventional mobile phone).Alternatively, a single camera with sufficiently fast focus could beused to obtain the same images by switching focus quickly between thesubject and background, or between any two or more objects at differentfocus distances, again depending on the speed of the auto focuscomponent of the camera. In the embodiments described above involvingscenes with four faces, the AF algorithm may be split across these fourdifferent face regions. The fast focus speed of an auto focus cameramodule that includes a MEMS actuator in accordance certain embodimentswould be divided among the four face regions so as to slow the autofocus for each face region by a factor of four. However, if thatreduction by four would still permit the auto focus to perform fastenough, a great advantage is achieved wherein video is optimized foreach of multiple subjects in a scene.

In a video embodiment, the camera is configured to alternate focusbetween two or more subjects over a sequence of raw video frames. Priorto compression, the user may be asked (or there may be a predetermineddefault set for a face before starting to record) to select a face toprioritize or a face may be automatically selected based onpredetermined criteria (size, time in tracking lock, recognition basedon database of stored images and/or number of images stored that includecertain identities, among other potential parameters that may beprogrammable or automatic. When compressing the video sequence, thecompression algorithm may use a frame with focus priority on theselected face as a main frame or as a key frame in a GOP. Thus thecompressed video will lose less detail on the selected “priority” face.

In another embodiment, techniques are used to capture video in low-lightusing sharp, underexposed video frames, combined with over-exposed videoframes. These techniques are used in certain embodiments for adaptingfor facial focus. In such an embodiment, the first frame in a videosequence is one with a focus optimized for one of the subjects.Subsequent frames are generated by combining this frame with 2nd, 3rd,and 4th video frames (i.e., in the example of a scene with four faceregions) to generate new 2nd, 3rd, 4th video frames which are “enhanced”by the 1st video frame to show the priority face with improved focus.This technique is particularly advantageous when large groups of peopleare included in a scene.

In a different context, such as capturing video sequences from the ridesat a theme park or social gatherings or baseball or soccer games, orduring the holidays, or in a team building exercise at the office, orother situation where a somewhat large group of people may be crowdedinto video sequences. The raw video sequences could be stored until avisitor is leaving the park, or goes to a booth, or logs into a websiteand uses a form of electronic payment or account, whereon the user cangenerate a compressed video that is optimized for a particular subject(chosen by the visitor). This offers advantageously improved qualitywhich permits any of the multiple persons in the scene to be the star ofthe show, and can be tremendously valuable for capturing kids. Parentsmay be willing to pay for one or more or even several “optimized” videos(i.e., of the same raw video sequence), if there are demonstrableimprovements in quality of each sequence at least regarding onedifferent face in each sequence.

Techniques Using Eye or Other Facial Sub-Region Information

Eye regions can be useful for accurate face focus, but as the eye isconstantly changing state it is not always in an optimal (open) statefor use as a focus region. In one embodiment a hardware templatematching determines if an eye region is open and uses this as a focusregion and the ISP applies a focus measure optimized for eye regions,and if the eye is not sufficiently open, then it defaults to a largerregion such as the mouth or a half face or full face and uses acorresponding focus measure.

In a portrait mode embodiment, a camera module may use multiple focusareas on specific face regions, e.g., two or more of a single eye, aneye-region, an eye-nose region, a mouth, a hairline, a chin and a neck,and ears. In one embodiment, a single focus metric is determined thatcombines the focus measure for each of two or more specific facialsub-regions. A final portrait image may be acquired based on this singlefocus metric.

In an alternative embodiment, multiple images are acquired, eachoptimized to a single focus metric for a sub-region of the face (orcombinations of two or more regions).

Each of the acquired frames is then verified for quality, typically bycomparison with a reference image acquired with a standard face focusmetric. Image frames that exceed a threshold variance from the referenceare discarded, or re-acquired.

After discarding or re-acquiring some image frames a set of differentlyfocused images remain and the facial regions are aligned and combinedusing a spatial weighting map. This map ensures that, for example, theimage frame used to create the eye regions is strongly weighted in thevicinity of the eyes, but declines in the region of the nose and mouth.Intermediate areas of the face will be formed equally from multipleimage frames which tend to provide a smoothing effect that may besimilar to one or more of the beautification algorithms described at USpublished patent application no. US20100026833, which is incorporated byreference.

Techniques employed to generate HDR images and eliminate ghosting insuch images, e.g., PCT/IB2012/000381, which is incorporated byreference, is advantageously combined with one or more of the fast autofocus MEMS-based camera module features described herein. The imagesutilized will include images with similar exposures, especially inportrait mode, while some of the exposure adjustment steps would beobviated in a portrait mode environment.

While an exemplary drawings and specific embodiments of the presentinvention have been described and illustrated, it is to be understoodthat that the scope of the present invention is not to be limited to theparticular embodiments discussed. Thus, the embodiments shall beregarded as illustrative rather than restrictive, and it should beunderstood that variations may be made in those embodiments by workersskilled in the arts without departing from the scope of the presentinvention.

In addition, in methods that may be performed according to preferredembodiments herein and that may have been described above, theoperations have been described in selected typographical sequences.However, the sequences have been selected and so ordered fortypographical convenience and are not intended to imply any particularorder for performing the operations, except for those where a particularorder may be expressly set forth or where those of ordinary skill in theart may deem a particular order to be necessary.

A camera module in accordance with certain embodiments includesphysical, electronic and optical architectures. Other camera moduleembodiments and embodiments of features and components of camera modulesthat may be included with alternative embodiments are described at U.S.Pat. Nos. 7,224,056, 7,683,468, 7,936,062, 7,935,568, 7,927,070,7,858,445, 7,807,508, 7,569,424, 7,449,779, 7,443,597, 7,768,574,7,593,636, 7,566,853, 8,005,268, 8,014,662, 8,090,252, 8,004,780,8,119,516, 7,920,163, 7,747,155, 7,368,695, 7,095,054, 6,888,168,6,583,444, and 5,882,221, and US published patent applications nos.2012/0063761, 2011/0317013, 2011/0255182, 2011/0274423, 2010/0053407,2009/0212381, 2009/0023249, 2008/0296,717, 2008/0099907, 2008/0099900,2008/0029879, 2007/0190747, 2007/0190691, 2007/0145564, 2007/0138644,2007/0096312, 2007/0096311, 2007/0096295, 2005/0095835, 2005/0087861,2005/0085016, 2005/0082654, 2005/0082653, 2005/0067688, and U.S. patentapplication No. 61/609,293, and PCT applications nos. PCT/US2012/024018and PCT/IB2012/000381, which are all hereby incorporated by reference.

Components of MEMS actuators in accordance with alternative embodimentsare described at U.S. Pat. Nos. 7,972,070, 8,014,662, 8,090,252,8,004,780, 7,747,155, 7,990,628, 7,660,056, 7,869,701, 7,844,172,7,832,948, 7,729,601, 7,787,198, 7,515,362, 7,697,831, 7,663,817,7,769,284, 7,545,591, 7,792,421, 7,693,408, 7,697,834, 7,359,131,7,785,023, 7,702,226, 7,769,281, 7,697,829, 7,560,679, 7,565,070,7,570,882, 7,838,322, 7,359,130, 7,345,827, 7,813,634, 7,555,210,7,646,969, 7,403,344, 7,495,852, 7,729,603, 7,477,400, 7,583,006,7,477,842, 7,663,289, 7,266,272, 7,113,688, 7,640,803, 6,934,087,6,850,675, 6,661,962, 6,738,177 and 6,516,109; and at

US Published Patent Applications Nos. US 2010-030843 A1, US 2007-0052132A1, US 2011-0317013 A1, US 2011-0255182 A1, US 2011-0274423 A1, and at

U.S. patent application Ser. Nos. 13/442,721, 13/302,310, 13/247,938,13/247,925, 13/247,919, 13/247,906, 13/247,902, 13/247,898, 13/247,895,13/247,888, 13/247,869, 13/247,847, 13/079,681, 13/008,254, 12/946,680,12/946,670, 12/946,657, 12/946,646, 12/946,624, 12/946,614, 12/946,557,12/946,543, 12/946,526, 12/946,515, 12/946,495, 12/946,466, 12/946,430,12/946,396, 12/873,962, 12/848,804, 12/646,722, 12/273,851, 12/273,785,11/735,803, 11/734,700, 11/848,996, 11/491,742, and at

USPTO-Patent Cooperation Treaty applications (PCTS) nos. PCT/US12/24018,PCT/US11/59446, PCT/US11/59437, PCT/US11/59435, PCT/US11/59427,PCT/US11/59420, PCT/US11/59415, PCT/US11/59414, PCT/US11/59403,PCT/US11/59387, PCT/US11/59385, PCT/US10/36749, PCT/US07/84343, andPCT/US07/84301.

All references cited above and below herein are incorporated byreference, as well as the background, abstract and brief description ofthe drawings, and U.S. application Ser. Nos. 12/213,472, 12/225,591,12/289,339, 12/774,486, 13/026,936, 13/026,937, 13/036,938, 13/027,175,13/027,203, 13/027,219, 13/051,233, 13/163,648, 13/264,251, and PCTapplication WO/2007/110097, and U.S. Pat. Nos. 6,873,358, and RE42,898are each incorporated by reference into the detailed description of theembodiments as disclosing alternative embodiments.

The following are also incorporated by reference as disclosingalternative embodiments:

U.S. Pat. Nos. 8,055,029, 7,855,737, 7,995,804, 7,970,182, 7,916,897,8,081,254, 7,620,218, 7,995,855, 7,551,800, 7,515,740, 7,460,695,7,965,875, 7,403,643, 7,916,971, 7,773,118, 8,055,067, 7,844,076,7,315,631, 7,792,335, 7,680,342, 7,692,696, 7,599,577, 7,606,417,7,747,596, 7,506,057, 7,685,341, 7,694,048, 7,715,597, 7,565,030,7,636,486, 7,639,888, 7,536,036, 7,738,015, 7,590,305, 7,352,394,7,564,994, 7,315,658, 7,630,006, 7,440,593, and 7,317,815, and

U.S. patent application Ser. Nos. 13/306,568, 13/282,458, 13/234,149,13/234,146, 13/234,139, 13/220,612, 13/084,340, 13/078,971, 13/077,936,13/077,891, 13/035,907, 13/028,203, 13/020,805, 12/959,320, 12/944,701and 12/944,662, and

United States published patent applications serial nos. US20120019614,US20120019613, US20120008002, US20110216156, US20110205381,US20120007942, US20110141227, US20110002506, US20110102553,US20100329582, US20110007174, US20100321537, US20110141226,US20100141787, US20110081052, US20100066822, US20100026831,US20090303343, US20090238419, US20100272363, US20090189998,US20090189997, US20090190803, US20090179999, US20090167893,US20090179998, US20080309769, US20080266419, US20080220750,US20080219517, US20090196466, US20090123063, US20080112599,US20090080713, US20090080797, US20090080796, US20080219581,US20090115915, US20080309770, US20070296833 and US20070269108.

What is claimed is:
 1. An auto focus camera module, comprising: a camera module housing defining an aperture and an internal cavity to accommodate camera module components; an image sensor coupled to or within the housing; a lens barrel within the housing that contains an optical train including at least one movable lens disposed relative to the aperture and image sensor to focus images of scenes onto the image sensor along an optical path; and a fast focus MEMS actuator comprising at least one electrostatic comb drive and coupled to one or more lenses of the optical train including the at least one movable lens and configured to rapidly move said at least one movable lens relative to the image sensor to provide autofocus for the camera module in each frame of a preview or video sequence or both, wherein the fast focus MEMS actuator is configured to alternately auto focus on two or more regions of interest, such that each region of interest is refocused every respective two or more frames of the preview or video sequence or both.
 2. The camera module of claim 1, wherein the fast focus MEMS actuator is configured to move the at least one moveable lens from a first position for a first one of the frames to a second position for a second, subsequent one of the frames to reliably refocus for the second subsequent frame within approximately 33 ms.
 3. The camera module of claim 1, comprising a face tracking module that is configured to predict a location of a face region of interest in a future frame permitting the auto focus camera module to focus on the region of interest quickly.
 4. The camera module of claim 1, comprising a face detection module that is configured to apply multiple short classifier chains in parallel to one or more windows of an image frame.
 5. The camera module of claim 1, wherein the two or more regions of interest comprise two or more sub-regions of a face.
 6. The camera module of claim 1, comprising a face recognition module that is configured to identify and prioritize one or more faces that correspond to one or more specific persons.
 7. The camera module of claim 1, wherein the fast focus MEMS actuator is further configured to move the at least one movable lens relative to the image sensor to provide local focus on multiple regions of interest within a common image frame.
 8. The camera module of claim 7, wherein the fast focus MEMS actuator is configured to cease lens motion during exposure of a first set of pixels in the common image frame, move the at least one movable lens between the exposure of the first set of pixels and exposure of a second set of pixels in the common image frame, and cease lens motion during the exposure of the second set of pixels.
 9. The camera module of claim 1, wherein the optical train comprises: first and second fixed lenses disposed on a first side of the at least one moveable lens; third and fourth fixed lenses disposed on an opposing second side of the at least one moveable lens; and wherein the fast focus MEMS actuator is formed from silicon or a substantially polymeric material and has a stroke of approximately 100 microns.
 10. A method, comprising: providing a camera module housing defining an aperture and an internal cavity to accommodate camera module components; providing an image sensor coupled to or within the housing; providing a lens barrel within the housing that contains an optical train including at least one movable lens disposed relative to the aperture and image sensor to focus images of scenes onto the image sensor along an optical path; providing a fast focus MEMS actuator including at least one electrostatic comb drive coupled to one or more lenses of the optical train including the at least one movable lens and configured to move said at least one movable lens relative to the image sensor to provide autofocus for the camera module in each frame of a preview or a video sequence or both; and operating the fast focus MEMS actuator to alternately auto focus on two or more regions of interest, such that each region of interest is refocused every respective two or more frames of the preview or video sequence or both.
 11. The method of claim 10, further comprising moving, with the fast focus MEMS actuator, the at least one moveable lens from a first position for a first one of the frames to a second position for a second subsequent one of the frames to reliably refocus for the second subsequent frame within approximately 33 ms.
 12. The method of claim 10, further comprising providing a face tracking module that is configured to predict a location of a face region of interest in a future frame permitting the auto focus camera module to focus on the region of interest quickly.
 13. The method of claim 10, further comprising providing a face detection module that is configured to apply multiple short classifier chains in parallel to one or more windows of an image frame.
 14. The method of claim 10, wherein the two or more regions of interest comprise two or more sub-regions of face.
 15. The method of claim 10, further comprising providing a face recognition module that is configured to identify and prioritize one or more faces that correspond to one or more specific persons.
 16. The method of claim 10, further comprising operating the fast focus MEMS actuator to move the at least one movable lens relative to the image sensor to provide local focus on multiple regions of interest within a common image frame.
 17. The method of claim 16, wherein the operating comprises: ceasing lens motion during exposure of a first set of pixels in the common image frame; moving the at least one movable lens between the exposure of the first set of pixels and exposure of a second set of pixels in the common image frame; and ceasing lens motion during the exposure of the second set of pixels.
 18. The method of claim 10, wherein the optical train comprises: first and second fixed lenses disposed on a first side of the at least one moveable lens; third and fourth fixed lenses disposed on an opposing second side of the at least one moveable lens; and wherein the fast focus MEMS actuator is formed from silicon or a substantially polymeric material and has a stroke of approximately 100 microns. 